1. Field of the Invention
The present invention relates to a method of image analysis and an apparatus for carrying out the method. The invention is particularly, although not exclusively, suited to surveillance applications.
2. Description of the Related Art
U.S. Pat. 4,249,207, issued to Computing Devices Company of Ottawa, Canada, relates to a surveillance system in which an image received by a television camera is divided electronically into an array of cells or `tiles`. The tiles are dimensioned to allow for perspective, so that they each cover substantially the same area regardless of the particular part of the image to which they respectively correspond. This is illustrated in FIG. 1 of the accompanying drawings. The system monitors each cell to determine whether a potentially significant event is occurring in each cell. If such an event is detected in one cell, neighbouring cells are checked and if an event is occurring in, say two, of these, it is taken as indicative of a track of a moving person or object of interest.
Occurrence of a potentially significant event in a cell is determined according to changes in the average light intensity received from that cell. Each cell consists of an array of pixels. These are analysed line by line and the integrated incident light intensity for that cell is computed. Over time, a weighted comparison is continuously made between the average present intensity and the average intensity in a previous frame. If the result of this comparison exceeds a predetermined threshold, then that is taken as a potentially significant event occurring in that cell.
The aforementioned U.S. patent states that this system provides good discrimination against false events such as moving cloud shadows. However, that system is best suited to generally empty scenes such as the area between two perimeter fences sited in an open landscape. It is much less able to discriminate against false events in a busy scene, such as may occur in an urban environment.
Specific disadvantages of this known system are:
a. It is overly sensitive to certain small variations in the image e.g. due to weather changes, clouds, tree movement, etc. This leads to a large number of false alarms resulting from normal changes in a scene; PA1 b. it has low sensitivity to certain large variations in the image, e.g., a small light change spread across a tile has the same effect as a large light change in part of a tile. This can cause some events of interest to be missed; and PA1 c. large changes in the image desensitise the system for a while. As a result, a return to a normal scene will cause an activity alarm, and this reduces the ability to track movement and identify the shape of objects, since tiles vacated by a slow object have the same activity as a tile being entered by the object. PA1 (a) processing the image signal to regard it as representing one or more cells each comprising a plurality of pixels; PA1 (b) for consecutive frames of the monitored image, generating characteristic values according to the detailed content of the image; and PA1 (c) for consecutive image frames, accumulating the characteristic values so that in a single operational mode, both a time history of the image, stored in a memory, is updated and an indication is produced indicative of the occurence or non-occurence of an event not associated with image background. PA1 arrival of an obstruction on a railway line, PA1 arrival or departure of a vehicle through a gate, PA1 arrival or departure of a person through a door, PA1 occurence of congestion on a railway platform. PA1 (a) processing the image signal to regard it as representing one or more cells each comprising a plurality of pixels; and PA1 (b) for consecutive frames of the monitored image, randomly or pseudo-randomly selecting pairs of pixels in the cell or cells, as appropriate, and for each pair, generating a binary signal in dependence upon whether or not the intensity of light incident on a predetermined one of the pair of pixels is greater than the intensity of light incident on the other. PA1 (a) processing the image signal to regard it as representing one or more cells each comprising a plurality of pixels; PA1 (b) for consecutive frames of the monitored image, generating characteristic values according to the detailed content of the image; and PA1 (c) generating a set of pointers a.sub.1, . . . , a.sub.m, each of n binary bits and sequentially setting these bits according to the characteristic values. PA1 (a) processing the image signal to regard it as representing one or more cells each comprising a plurality of pixels; PA1 (b) for consecutive frames of the monitored image, generating a set of pointers each containing a value related to the detailed content of the image; and PA1 (c) using the pointers to address a memory array and for successive frames, updating the value of an address in each row of the array indicated by the pointer corresponding to that row. PA1 (a) processing the image signal to regard it as representing one or more cells each comprising a plurality of pixels; PA1 (b) for consecutive frames of the monitored image, updating one address in each row of a memory array in dependence on the detailed content of the image; and PA1 (c) over all the rows, summing the values in the addresses being updated and summing the maximum value in each row, subjecting the ratio of the two sums to a time weighted averaging with respect to the ratios obtained from previous frames, comparing the ratio with the time weighted average and using the result to determine the occurence or non-occurence of a significant event within the image.
The above problems explain why this existing system is only effective in scenes with very little variation in light levels. In fact it is only effective for monitoring indoor scenes or outdoor scenes where there is no expectation of movement, e.g., dead zones between fences surrounding a prison.
The present invention in its various aspects has a significant advantage over that described in the aforementioned U.S. patent in that it provides the ability to `learn` the nature of a complex backround. This enables it to discriminate better for genuine events of interest in an otherwise crowded or busy scene.
Another previously known system called `WISARD` is described by I. Aleksander et al in Sensor Review, July 1984, pp 120-124. Pixels within a cell are interrogated pseudo-randomly. In the latter disclosure it is stated that the intensity of light incident on a pixel can be converted to a Grey-scale signal or alternatively, binary signal.
In the latter case, the detected light level is compared with a threshold and a 1 or 0 is set according to whether the intensity is below the threshold or not. These binary signals are then combined into groups, which form pointers into an array stored in a random access memory (RAM). Each time an array element is addressed by a pointer, its value is set to "on" otherwise it is left unchanged. This process is carried out over a number of "frames".
Thus, with WISARD, over successive frames, values in certain elements of the array become set. This is in effect a `training period`. After sufficient frames have elapsed, the array is considered suitably primed and the system is put into run mode and pointers are generated in the same way. Then, if a pointer addresses an element which is set, this indicates that the intensities in pixels forming the pointer are similar to values which occurred in the training set. Pointers which address elements which are not set indicate that the situation is different from the training period. In this way WISARD detects when the scene changes.
It will be apparent that WISARD can only compare the present situation with what happened during its finite `training` period and has no memory for events before or since. Also, transient events in the training period have as much effect as common events. As a result, WISARD cannot continuously improve its record of background, and so is limited in its ability to discriminate between significant events and the backround.
The present invention is capable of continuously improving its record of the background and so can provide better discrimination than WISARD.
A first aspect of the present invention provides a method of analysing an electronic image signal, the method comprising:
The first aspect of the invention also includes a device for analysing an electronic image signal, the device comprising electronic means for effecting steps (a)-(c) in the aforementioned method.
The present invention in general, mitigates the problems of the prior art referred to above and can extend the possibilities for scene monitoring to a wider variety of scenes. In particular, the invention is suited to monitoring scenes with a large amount of normal activity, for example to monitor busy street scenes to identify specific events. Examples of such events are:
Although in its widest definition, the first aspect of the invention requires only one cell, it is preferred that the image is represented as a plurality of cells, most preferably adjusted for perspective as hereinbefore described.
A preferred method of generating the characteristic values is to randomly or pseudo-randomly select pairs of pixels and for each pair, generate a (preferably binary) signal in dependence upon whether or not the intensity of light incident on a predetermined one of the pair of pixels is greater than the intensity of light incident on the other.
Thus, a second aspect of the present invention provides a method of analysing an electronic image signal, the method comprising:
The second aspect of the invention also includes a device for analysing an image signal, the device comprising electronic means for effecting steps (a) and (b) of the method according to the second aspect of the invention.
Preferably at least one pair of pixels is so selected for each of a plurality of cells. It is also much preferred to select in the same way, a plurality of pixels for each cell of the image field.
Once one or more pairs of pixels are selected in a given cell, the same pixels can then be used in subsequent frames until the end of the operating session. -In a subsequent session, the same pairs could be used again or different pairs selected for the remainder of that session.
When a plurality of pixel pairs are used in a given cell, it is possible for the same pixel to be used in more than one pair, although this is not absolutely necessary. It is also possible, although not mandatory, for one or some of the pixels not to be used in any pair.
For different cells, the pair or pairs of pixels could be chosen at the same relative positions in each although again, this is not necessary.
We also prefer utilising the characteristic values to generate pointers for addressing the memory containing the time history of the image. A particularly advantageous means of doing this is to generate a set of pointers a.sub.1 . . . , a.sub.m, each of n binary bits and sequentially setting these bits according to the respective binary values of the signals resulting from the comparisons of the pixel pairs. In other words, pairs 1, . . . , n of the pixel pairs are used to set the values of a [1, . . . , n] then pairs (n+1), . . . , 2n are used to set a.sub.2 [1, . . . , n] and so on.
Therefore, a third aspect of the present invention provides a method of analysing an electronic image signal, the method comprising:
The third aspect of the present invention also includes a device for analysing an image signal, the device comprising electronic means for effecting steps (a)-(c) of the method according to the third aspect of the invention.
The values of n and m may be any convenient and may be the same or different, but n.times.m will be equal to the number of characteristic values. Therefore, in the context of the second aspect of the invention, n.times.m will be equal to half the number of pixels selected in each cell.
In a preferred embodiment, these pointers are used to address a memory comprising an array of addresses comprising rows 1, . . . , m and columns 1, . .. ,2.sup.n. When the characteristic values have been used to set the values of the pointers, the latter address the memory as follows. Pointer a.sub.1 addresses row 1. The pointer contains a binary number of value somewhere between 0 and 2.sup.n. The pointer points to the column in row 1 having a column number corresponding to the value in a.sub.1. The value in that address is then incremented by 1 (at the outset, all addresses will be set to zero). This is done for all rows up to m. In the next frame, the whole process is repeated, updating the value by 1 in a particular address pointed to in each row.
Therefore, a fourth aspect of the present invention provides a method of analysing an electronic image signal, the method comprising:
The fourth aspect of the present invention also includes a device for analysing an image signal, the device comprising electronic means for effecting steps (a)-(c) of the method according to the fourth aspect of the invention.
In a preferred embodiment, the set of pointers comprises pointers a.sub.1, . . . , a.sub.m, each of n binary bits and the memory array has m rows and 2.sup.n columns.
It will be appreciated that in terms of a memory array, the terms `rows` and `columns` are purely arbitrary and therefore may be transposed.
A further advantageous feature of the invention is the way in which the values in the memory are used to discriminate between the background and unusual or unexpected occurences. In successive frames, for each row in the memory, the maximum value at any address in the row is determined, the value currently pointed at also being known. For all the rows, the ratio of the sum of values in currently pointed to addresses, divided by the sum of the maxima is calculated. This ratio is compared with a time weighted average for the same ratio from previous frames. If this ratio is less than a chosen proportion of the time weighted average then it is taken as indicative of an unusual or unexpected event occuring in the relevant cell.
Thus a fifth aspect of the present invention provides a method of analysing an electronic image signal, the method comprising:
The fifth aspect of the present invention also includes a device for analysing an image signal, the device comprising electronic means for effecting steps (a)-(c) of the method according to the fifth aspect of the invention.
When the present invention in any of its aspects is manifested as a device as opposed to a method, the device may be implemented as hardware/firmware arranged to operate under the control of appropriate software or it may be realised in terms only of hardware/firmware configured from logic gates, flip-flops etc, arranged to carry out the required functions.